The background description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent the work is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.
De-interlacing addresses a legacy problem: interlaced video required by old cathode ray tube (CRT)-based analog televisions must be converted to be shown on today's digital televisions. An interlaced video is a succession of 50/60 fields per second, where each field carries only half of the rows that are displayed in each frame of video. Much of today's video content is available in the interlaced format and requires de-interlacing since newer liquid crystal display (LCD) and plasma based displays require progressive video input.
In interlaced video, one frame of video is broken into two fields, one of which contains even lines and the other odd lines. To display interlaced video on newer LCD or plasma display, the display must be de-interlaced. The newer displays are progressive in that each frame comprises a set of pixels (e.g., 1920×1080). Two fields contain pixels of one frame. Each field records pixels that are separated in time. Assuming there are 30 frames per second (fps) or 60 fields per second, field 0 is at time t, and field 1 is at time t+ 1/60. Since the fields are recorded at slightly different time intervals, the two fields cannot be combined to create a progressive frame for any video that has motion. De-interlacing is complex due to the need to estimate and compensate for the potential motion in that one-sixtieth of a second.
Fundamentally, de-interlacing is a process of converting a stream of interlaced frames into a stream of progressive frames. Two basic de-interlacing methods are called bob and weave. In bob de-interlacing, each field becomes its own frame of video, so an interlaced National Television System Committee (NTSC) clip at 29.97-fps stream becomes a 59.94-fps progressive. Since each field has only half of the scan lines of a full frame, interpolation must be used to form the missing scan lines. Bob de-interlacing can also be called spatial line doubling, in which the lines in each field are doubled. The new line generated can be a copy of the previous line (scan-line duplication) or an average of the lines above and below (scan-line interpolation). Bob de-interlacing provides a good result when the image intensity varies smoothly, but it can soften the image since it reduces the vertical resolution.
Weave de-interlacing involves weaving two fields that are separated in time into one frame. Weave de-interlacing provides good results if there is no motion in the one-sixtieth of a second that separates the two fields (for NTSC video). Sometimes, when pairs of interlaced fields have been created from original progressive frames, the results of the weave algorithm are perfect. If there is motion, however, artifacts called mouse teeth appear.
Both bob and weave de-interlacing can affect the image quality, especially when there is motion. Bob de-interlacing can soften the image while weave de-interlacing can create jagged images or mouse teeth artifacts. Bob and weave de-interlacing can be combined to improve the quality of de-interlacing, where motion between successive frames of video is detected and computed. This combination technique, which uses the weave technique for static regions and the bob technique for regions that exhibit motion, is called motion-adaptive de-interlacing. The key to motion-adaptive de-interlacing is obtaining accurate motion detection and motion value calculation, usually by comparing pixels from one frame to the next.
Interlaced video can have complexities other than transmission of odd and even fields. For example, motion picture photography is progressive and is based on 24 fps while NTSC format is 60 fields per second. So conversion of motion picture photography into interlaced video can create complex cadences. To convert motion picture photography to interlaced video, each progressive frame is converted into two fields. So 24 fps converts to 48 fields per second. To increase the 48 fields to the required 60, a 3:2 pulldown technique or cadence is used to generate three fields from one film frame and two fields from the other film frame. In addition, sometimes every twelfth field is dropped to accelerate the film and fit it within a given time slot. This loss is barely noticed by an average viewer but results in a 3:2:3:2:2 cadence (or simply 3:2 pulldown), which includes the following sequence: Frame 1: 3 fields, Frame 2: 2 fields, Frame 3: 3 fields, Frame 4: 2 fields, Frame 5: 2 fields. The sequence is repeated.
In countries that use the Phase Alternating Line (PAL) or Sequential Couleur Avec Memoire (SECAM) video standards, films destined for television are photographed at 25 fps. Theatrical features originally photographed at 24 fps are shown at 25 fps. To convert 24 fps material to 25 fps, a 2:2:2:2:2:2:2:2:2:2:2:3 (Euro) pulldown (or simply 2:2 pulldown) is used, where a pulldown field is inserted every 12 frames, thus effectively spreading 12 frames of film over 25 fields (or 12.5 frames). The 2:2 pulldown is also used to transfer shows and films photographed at 30 frames per second to NTSC video, which has 60 Hz scanning rate. Additionally, professional camcorders and various types of video processing use different types of cadences. Accordingly, de-interlacers must compare incoming fields and detect the cadence. If esoteric cadences are not detected, video data may be unnecessarily discarded. In some instances, one part of a frame may have 3:2 cadence while another part may be straight interlaced (e.g., a film is inserted in an interlaced video). To detect and correctly de-interlace such a source would require de-interlacers to implement a per-pixel cadence detection.